Corporations, institutions, and governments spend hundreds of millions of dollars each year to digitize documents, films, maps, books, and other physical media. Included in this mix are billions of pages of medical records, legal evidence, corporate documents, material from national and regional archives, and banking checks. The resulting digital image files represent valuable information whose accuracy has significance in current working operations and for long-term archiving. The digitization process is the gateway for this information onto networked systems, which allows for convenient, cost effective, and efficient transmission, storage, searching, and retrieval of the image information.
Organizations also spend vast amounts of money on capturing day-to-day activities with digital image capture devices, such as inspection cameras for manufacturing processes, forensic crime-scene cameras, in-car police cameras, automated teller machine (ATM) cameras, and surveillance cameras for monitoring facilities, equipment, and personnel. Some applications use computer vision techniques to automatically analyze the images for certain features or events. In many cases, the images that are produced by these digital acquisition devices are never viewed by a human unless a specific event triggers a review. However, regardless of whether the images are analyzed by computers or viewed by humans, it is essential that the image data represents the physical scene with sufficient fidelity for the intended application.
Because of the sensitive nature of the information in many applications, it is important to ensure that image data is not tampered with after it is generated. It is a simple matter to change the contents of a digital image by using an image editor or other readily available computer technology. One approach to ensuring data integrity is to use encryption. However, encryption can be computationally expensive for large amounts of data, such as is the case for high resolution images and video sequences.
As a result, a more practical approach to ensuring the integrity of a digital data file is to use a digital signature. Digital signatures are based on the concept of a hash. A hash is a relatively short numerical value that represents a distilled version of the larger digital data file. Methods that perform this distillation are referred to as hash functions or hash algorithms, and hash functions are designed so that a small change in the digital data file will produce a significant change in the calculated hash value. A digital signature is an encrypted version of the hash, and the digital signature is associated with the digital file in some way, such as attaching it to the file header or storing in a database that is indexed by a unique identifier. An image that has been associated with a digital signature in the manner just described is often called a “secure” image. Tampering with the digital data can be detected by recalculating the hash and comparing it to the original hash in the secure digital signature. A benefit of securing images with digital signatures is that the image data itself is in the “clear”, that is, unencrypted, which means a secure image can be used like any other image, yet its integrity can be verified at any time.
While encryption and digital signatures allow the integrity of image data to be verified, they do not address the issue of the quality of the image data. Image quality is determined by many factors, including such attributes as resolution, sharpness, dynamic range, noise, and color reproduction. The digital image data that represents a physical medium or scene could be meaningless, erroneous, or artifact-laden for a variety of reasons, such as a scanner that is defective or a camera that is out of focus, for example. In such cases, the techniques for authenticating data as described previously may be of limited value because they may be protecting data that is worthless.
The knowledge that image data is a satisfactory replica of an original physical medium or scene is clearly important. Companies that are responsible for the scanning of important documents for governments, financial institutions, and other concerns may become liable for loss of valuable information if the scanned image quality is insufficient and the original physical documents have been destroyed. Even if the original documents are still available, significant costs may be incurred in rescanning. End users of scanned documents may also be affected by poor quality because of a diminished ability to extract or interpret the information that was contained in an original document. Likewise, law enforcement agencies may be hampered in their identification and prosecution of criminals if surveillance video images have insufficient quality.
In the U.S. banking industry, the Check Clearing for the 21st Century Act (“Check 21”) allows banks to move checks electronically, rather than as physical documents, in order to make the check clearance process faster and more efficient. A bank can scan a check and then transmit image data and payment information in lieu of sending the original check. Banks are not required to keep the original check, and it is typically destroyed or “truncated” to reduce maintenance costs. However, banks that scan checks under Check 21 are liable for any financial losses associated with poor quality images.
As a result, image quality is typically assessed at the point of image capture in a Check 21 environment, and the image quality affects the workflow of the electronic check data. For example, a poor quality image may require special handling, which incurs extra costs. A bank that receives a poor quality check image might require the originating bank to rescan the check, or the receiving bank might simply assume liability for the cost of the check if it is a small dollar amount. The result is increased service costs and delays in completing check clearance, as well as the potential loss of good will with customers. Thus, there is significant value associated with the ability to properly assess image quality.
There are various ways to assess image quality. One approach is to have a person review an image for image quality. However, given the tremendous number of images that are produced daily, a human-based quality control solution is not economically viable in many applications. In addition, human error rates may be significant due to various factors, such as fatigue and lack of training.
Another approach to assessing image quality is to use test targets. A test target acts as a reference image, and quality metrics calculated from that reference can provide measures of actual versus ideal performance for a capture device. Quality measurements using known test targets are termed “full reference” measurements. Test targets are often used on an intermittent basis during the operation of an image capture device to determine if the device is performing as expected. However, the intermittent use of test targets doesn't necessarily provide information about the image quality that is achieved for the capture of a particular physical medium or scene. In some applications, it may be possible to include a test target in every image that is captured by a device, but this can be costly and often impractical. Moreover, it still may be the case that quality of the captured medium or scene is not fully reflected in the quality that is determined from the included test target data. For example, an adaptive image processing algorithm that automatically controls image brightness and contrast might not produce the optimal code values for the captured medium or scene because of the background color in the image, while a test target may still be rendered appropriately.
A third approach is to assess image quality directly from the captured image data itself. When the only information that is available to assess quality is the image data, which generally has unknown characteristics, the quality measurement techniques are referred to as “no-reference” methods. An example of a no-reference image quality metric is described in a technical paper entitled “A no-reference perceptual blur metric” by P. Marziliano, F. Dufaux, S. Winkler, and T. Ebrahimi, Proceedings of the IEEE International Conference on Image Processing, Vol. III, pp. 57-60, September 2002. The method in this paper computes a blur metric (that is, a loss in sharpness) by identifying vertical edges in an image and then determining the average spatial extent of the edges. The Financial Services Technology Consortium (FSTC), which is a consortium of banks, financial services providers, academic institutions, and government agencies, has investigated a similar no-reference blur metric for Check 21 applications. The FSTC has also investigated a number of other no-reference quality metrics for Check 21 applications, including compressed image file size, document skew angle, and number of black pixels (for a bi-tonal image). A full description of the FSTC quality metrics can be found at the www.fstc.org Internet address (currently www.fstc.org/docs/prm/FSTC_Image_Defect_Metrics.pdf.)
Regardless of the method that is used to assess image quality, it is advantageous to have the image quality measures secured against possible tampering because of the previously discussed economic, liability, and legal issues that are associated with image quality. Moreover, at various points in the lifecycle of a digital image, it may be desirable to check quickly on the image quality without having to perform another visual inspection or computer analysis of the image data. This capability can be achieved by assessing image quality once (typically at the point of capture) and then securing the quality metrics against tampering. Furthermore, it is desirable to have the secure image quality measures and the secure image data be linked together so that any change in the image data renders the associated quality metrics as invalid.
Current applications that assess image quality, such as Check 21 processing systems, do not secure the image quality metrics and hence are susceptible to tampering of the quality data, which may result in an inefficient workflow and financial losses. It is easy to imagine that a digital scan of a check may be vulnerable to courtroom challenge on the basis of image quality, despite the use of digital signatures for the image data itself by the bank. With secure image quality measures, the liabilities of those parties who are responsible for the scanned data are minimized.
In commonly assigned co-pending U.S. patent application Ser. No. 11/454,673 to McComb, noted earlier, a method is taught for measuring the scanned image quality of documents using test targets and for securing the image quality measurements in combination with secure image data. The document images that are produced by this method are termed “assured documents”. Image quality metrics are calculated from test targets that are periodically inserted into a document queue, and these metrics are associated with the scanned image data for user documents that are in the same document queue. If the quality metrics meet predetermined quality specifications, the quality metrics are associated with the image data of an individual user document by combining the quality metrics with a secure hash value that represents a distillation of the image data, followed by encryption of the combined quality metrics and hash value. The encrypted quality metrics and hash value are stored in the file header or filename of the digital document, or by other means, as disclosed in the co-pending application by McComb, to produce an assured document. If the quality metrics do not meet predetermined quality specifications, an assured document is not produced.
In a commonly assigned co-pending U.S. patent application Ser. No. 11/940,347 to Honsinger, et al., noted earlier, improvements are taught for the method by McComb. One improvement is the use of no-reference quality metrics, as described previously, which reduces or eliminates the need for test targets to assess image quality. This is advantageous in applications where test targets are not readily available, economically viable, or otherwise usable.
Another improvement in the method by Honsinger et al. is the concept of an “assured document” is extended to provide for an “assured image”, which refers to image data that has been processed so that (1) any tampering with the image data can be detected, (2) the image quality of the image data has been measured and the image quality metrics have been secured, and (3) the image quality metrics are linked to the image data so that any changes to the image data render the image quality metrics as invalid. The secure assurance of all images, regardless of whether their image quality meets predetermined quality specifications, provides increased utility as compared to the assurance of images only when the quality is found to be sufficient, as was the case in the method by McComb. As an example, a digital video image sequence from a police surveillance camera may have some frames that have excellent quality, while other frames in the same sequence have poor quality. However, every frame in the sequence may be essential as evidence, and hence it is imperative to secure the image data and the quality measurements in every frame, regardless of the image quality.
Both the method by McComb and the method by Honsinger et al. use quality thresholds that are applied against individual quality metrics to determine if image quality is sufficient for an intended application. This approach can be problematic with image capture devices that operate in dynamic environments, such as with an outdoor surveillance camera, where weather and lighting conditions will change frequently. Thresholding against individual quality metrics may also result in some images being accepted as having sufficient quality, when in fact a combination of image degradations produces insufficient quality. For example, both sharpness and noise may be within acceptable limits on individual bases, but the combination of these two degradations can result in poor quality.
In addition, both the method by McComb and the method by Honsinger et al. assess image quality according to only two classes, namely, sufficient for, or insufficient for, an intended application. The use of additional quality classes can be beneficial in some applications, but the classification of quality by using multiple thresholds applied to multiple quality metrics is difficult to accomplish in an efficient and robust manner.
As mentioned previously, the method by Honsinger et al. produces an assured image regardless of whether the image quality was assessed to be sufficient or insufficient. However, the assessed quality class is not included as part of the secure assurance data, which can be inconvenient and computationally inefficient as it requires the secure quality metrics to be reevaluated any time the assessed quality class is required.
Thus, there is the need for a method to (i) robustly and efficiently assess the image quality of image data without relying on thresholding of individual quality metrics, (ii) provide for quality assessments with an arbitrary number of quality classes, and (iii) secure the assessed quality class information when forming an assured image.